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Abstract 


This paper focuses on a trend to analyse temporal characteristics of constructs important to 
learning and instruction. Different researchers have indicated that we should pay more attention to 
time in our research to enhance explanatory power and increase validity. Constructs formerly 
viewed as personal traits, such as self-regulated learning and motivation, are now conceptualized 
as a series of events that unfold over time. This raises new questions with regard to the temporal 
characteristics of these constructs and their dynamic interplay with learner and context 
characteristics. Even though the value of analyzing temporal characteristics is becoming evident, a 
number of challenges need to be tackled in order to make progress in the field of learning and 
instruction. First, we need to be aware of the paradigm shift that temporal analysis entails. Second, 
a common understanding of different dimensions of time and the position of temporal 
characteristics therein can facilitate our time-related research dialogue. Third, a better 
understanding how to answer time-related questions with appropriate methodological approaches 
needs to emerge. Fourth, researching temporal characteristics requires procedures and guidelines 
for segmenting time units. Fifth, temporal data are mostly collected at the micro level, whereas 
most theory is defined at a macro level; consequently we need to bridge these differences in the 
granularity used betw’een collecting, coding and theorizing to enhance meaning making. Finally, 
so far, most examples of time-related research are exploratory or comparative studies; the next step 
is to move toward confirmative studies, which constitute the “Holy Grail ” of temporal analysis. 
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1. Introduction 

Learning is defined as the acquisition of skills and knowledge and can be recognized through 
changes in the learners’ behaviour (Mayer 2008; Zimmerman 2002). The concept of time is innate to 
learning, as it takes time to acquire skills and knowledge and to signal changes in behaviour. In learning and 
instruction research, we mostly capture time in pre- and post-test designs. As such we often focus on a 
narrow concept of time, reducing the temporal characteristics of learning to the changes between pre- and 
post-tests which reduces validity and explanatory power of our research. Currently, technological 
advancements increase our ability to gain traces of learners while they are learning, which is an important 
facilitator to overcome this limited focus on time in our field (Greene & Azevedo, 2010; Reimann, 2009; 
Winne, 2010). A steadily growing group of researchers is raising questions that address how different 
constructs act and develop over time (Bannert et al., 2014, Greene & Azevedo, 2010; Molenaar & Chiu, 
2014; Riemann, 2009; Schmitz, 2006; Wise & Chiu, 2011). With this growing interest in temporal 
characteristics of constructs at the heart of learning and instruction research, the need for temporal analysis is 
becoming more prevalent. 

This paper focuses on the developing trend in learning and instruction research to analyze temporal 
characteristics of different constructs. The rationale for temporal analysis in our field is discussed as well as 
the fact that temporal analysis entails a deviation from our main research approach, changing our analysis 
from characteristics of students to attributes of. learning activities (Riemann, 2009; Schmidt, 2006). 
Researchers have conceptualized temporal characteristics of learning and instruction constructs in many 
ways in their research, leading to a diverse set of dimensions of time driving research questions. It is argued 
that a conceptual framework of temporal characteristics can support transparency and enhance comparability 
in the field. Lastly, a number of challenges are discussed that we, as a field, need to overcome to successfully 
engage in temporal analysis. 


2. The rationale for temporal analysis 

A number of researchers in the field of computer-supported learning (Kapur, 2011; Reiman, 2009;) 
and self-regulated learning (Greene & Azevedo, 2010; Schmitz, 2006; Schoor & Bannert, 2012) indicate that 
we should pay more attention to time in the learning process. Existing research methods do not “fully” utilize 
the temporal information embedded in the data collected (Kapur, Voiklis & Kinzer, 2008; Wise, Perera, 
Hsiao, Speer & Marbouti, 2012). This reduces the explanatory power of the analysis performed and limits 
the validity of the conclusions drawn (Akhras & Self, 2000; Reimann, 2009). For example, Kuvalja and 
colleagues (2014) show that self-directed speech and self-regulatory behaviour of children with a specific 
language impairment does not differ in frequency; neither the number of self-directed speech and self- 
regulatory events during learning, nor the order between the these events as detected by sequential lag 
analysis differed, but there was a difference between the two groups in the co-occurrence of self-directed 
speech and self-regulatory behaviour as detected by temporal patterns analysis (Magnusson, 2000). This 
indicates that without proper temporal analysis, existing differences between groups of learners cannot be 
detected. 

Moreover, a number of constructs, such as self-regulated learning and motivation, that were 
traditionally viewed as a trait of the learner are now conceptualized as a series of events (Bannert et al. 2014; 
Greene & Azevedo, 2009; Schmitz, 2006). Driving this conceptual change are indications that self-report 
data have little relation with the actual student behaviour during learning (Veenman, 2011). These findings 
point towards the need for new conceptualisations of these constructs. A temporal conceptualisation viewing 
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self-regulated learning as a series of events that act differently over time and changing contexts, might 
overcome these issues (Azevedo et al., 2010; Hadwin & Jarvela, 2011). For example, Malmberg and 
colleagues (2014) show that students use different strategies and learning patterns when working on an ill- 
structured task compared to a well-structured task. A series of events can be perceived as a process that 
unfolds over time in a certain order (Reimann, 2009). For example, self-regulated learning processes of 
successful students show a cyclical order among different strategies that repeat over time (Bannert et al. 
2014). Moreover, Molenaar and Chiu (2014) found strong positive predictive relations between different 
learning activities during collaborative learning over time. The changed conceptualization of constructs 
raises new questions with regard to the characteristics of these constructs and their dynamic interplay with 
the learning context. 

Finally, an emerging interest is in connecting different levels of analysis (Hollenstein, 2013; Suthers, 
Teplovs, de Laat, Oshima & Zeini 2011). This investigates of how macro-level phenomena can emerge from 
and/or be constrained by different micro-level dynamics. For example, Chiu (2008) found that micro¬ 
creativity in a group’ mathematical solutions can be sparked by a discourse pattern, namely a wrong idea 
followed by disagreement among the group members. Temporal analysis can help develop an understanding 
of how patterns unfold, providing insights into how learning is taking place (Chiu, 2008; Wise & Chiu, 
2013). 

Take together, the argument for temporal analysis is driven by the realisation that without careful 
attention for temporal characteristics of constructs in learning and instruction research, we are reducing the 
significance of our research and are unable to explain important aspects of learning and instruction. 


3. A paradigm shift 

As touched upon in the introduction, it is important to understand that advanced temporal analysis 
entails a deviation from the traditional research paradigm used in learning and instruction (Reimann, 2009; 
Schmitz, 2006). Often the variable-based approach is applied, which focuses on the analysis of variance 
between independent and dependent variable(s). In contrast, the event-based approach looks at events 
analysing the (dynamic) relations between the events (Reimann, 2009). This approach focuses on 
researching the nature of these relations and their development over time. This reveals the temporal 
characteristics of a construct and/or how different constructs interplay over time. For example, it can indicate 
how a discussion among learners unfolds over time. Consistency and change in the behaviour of constructs 
can be investigated by specifying these temporal characteristics (Schmitz, 2006). 

Yet often reviewers in learning and instruction immediately ask the next question: can we explain 
learning performance from temporal characteristics of the constructs? This question embodies the ’’Holy 
Grail” of temporal analysis and often constitutes a connection between our traditional variable-based 
approach and the event-based approach. However, few (perhaps none) researchers have so far reached the 
“Holy Grail”. Moreover, many of those initially aiming for this connection started to grow a realization that 
there are valuable questions to be answered within analysing temporal characteristics themselves. An 
example of such a research question is: Which sequences of learner actions (discuss, elaborate, summarize) 
occur during collaborative learning? An example of a research question combining temporal characteristics 
with learning performance is: Which sequences of learners’ actions during collaborative learning influence 
learning performance positively? Overall temporal analysis in learning and instruction is innate to our 
intuitive understanding of learning, but the operationalization of this understanding entails a deviation of our 
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traditional research paradigm. Consequently, the nature of the questions addressed with temporal analysis 
varies from our characteristic research questions in learning and instruction. 

After all, Time is a highly complex construct that has been debated on from physics to philosophy. 
Also within educational research, conceptualizations of different time scales (Lemke, 2000) and the use of 
time in classrooms (Bloome et al., 2009; Mercer, 2008) have been discussed. Still, there is no framework that 
conceptualizes dimensions of time and positions different temporal characteristics within these dimensions. 
Research questions, therefore, focus on different dimensions of time and address conceptually different 
temporal characteristics of constructs. In the next section, different dimensions of time important for learning 
and instruction research are highlighted. 


4. Different dimensions of time 

So far, in our field when addressing temporal characteristics, we have encountered mainly frequency 
analysis indicating the number of occurrences of a variable during a particular time window. This provides 
insights into the prevalence of a construct during learning. For example, students receiving scaffolds during 
learning apply more metacognitive activities compared to students that do not receive scaffolds (Molenaar et 
al., 2011). Although informative, frequency analyses provide limited insights into the individual time-related 
characteristics of the constructs researched. Even though this analysis showed that students perform more 
metacognitive activities, we do not know the importance of their position in the learning process, their 
duration or the rate at which these metacognitive activities occur during learning. Thus frequency analyses 
treat the learning process as one holistic unit, ignoring the individual time-related characteristics of 
constructs. Using the individual time-related characteristics allows for the analysis to illustrate how events 
occur within the flow of continuous events in a particular time window. Examples are analyzing the 
significance of the position of events, the duration of particular events and the rate of particular events within 
the learning process (Molenaar & Wise, in prep). For example, planning at the start of a learning task was 
found to be more productive for learning compared to planning latter on (Moos et al. 2008). Also students 
monitor at a higher intensity and longer in more difficult tasks compared to easier tasks (Iiskala et al. 2010). 
The dimension of time described above conceptualizes how constructs behave in a continuous flow of events 
by examining the individual time-related characteristics of these events within the flow. 

Another dimension of time in contrast to analysing events in a continuous flow, is analysing relative 
arrangements of multiple events in time. Here the focus does not lie on the individual time-related 
characteristics of events in a time window, but on how events are organized among each other. Examples are 
both reoccurring processes and non-reoccurring processes (Molenaar & Wise, in prep). An example of a 
reoccurring process is the cyclical notion of self-regulated learning, which suggests that orientation, 
planning, monitoring and evaluation follow each other (Hadwin & Jarvela, 2013; Zimmerman, 2002). Non- 
reoccurring transitions occur only once, for example, students who learn how to read progress from spelling 
letters into the automatic detection of words (Verhoeven, 2004). Apart from reoccurring and non-reoccurring 
patterns which both indicate a form of regular change, irregular change is another form of an arrangement of 
multiple events that can be investigated. The notion of productive failure where collaborating students seem 
to engage in chaotic interaction in the beginning of their collaboration is an example of irregular change 
(Kapur, 2009). This seemingly unstructured process is of essential importance for their later learning. The 
dimension of time described above conceptualizes how constructs behave in in relative arrangements of 
multiple events by examining the organisation among these events. 
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Without claiming that the above is a complete overview of temporal characteristics useful for the 
field of learning and instruction, a clear distinction can be made between two dimensions of time, i.e. 
focusing on individual events within the continuous flow of events or on relative arrangements of multiple 
events (Molenaar & Wise, in prep). In order to push the conceptual understanding of time in our field, a 
conceptual framework of looking at time and positioning temporal characteristics therein is important for 
learning and instruction research to articulate and classify time-related research questions. Such a framework 
can support conceptual clarity among researchers engaging in temporal analysis and organize and deepen 
debates. Furthermore, it can be used as a roadmap to articulate temporal research questions, unravelling 
temporal characteristics of different constructs. 


5. An illustrative example of temporal analysis 

In order to illustrate the above, I provide an example of a temporal analyses used to research socially 
regulated learning. During collaborative learning, students support one another’s learning as they discuss, 
elaborate, argue, confirm and regulate one another’s activities. We know that regulative activities such as 
metacognitive (i.e., planning, monitoring) and relational activities (i.e., confirming, engaging) contribute 
significantly to students’ learning (Molenaar et al., 2011). Yet, we know very little about how the group’s 
socially regulative activities influence students’ cognition at a micro level during collaborative learning. 
Therefore, we explored how sequences of students’ cognitive, metacognitive and relational activities affect 
the likelihood of subsequent cognitive activities during collaborative learning and whether these 
relationships differ across time (Molenaar & Chiu, 2014). 

The data are from 18 triads (54 students) engaged in 51.338 conversation turns over 6 hours of 
learning activities. The triads collaborated face-to-face while working in a computer based learning 
environment. The primary school students were in grades 4, 5, and 6, and aged between 10 and 12. Statistical 
discourse analysis, content and discourse analysis were used to analyse the learning activities. During 
content analysis, each turn in the conversation was coded as cognitive (higher or lower cognition), 
metacognitive (orientation, planning, monitoring and evaluation) or relational (confirm, deny, engage), 
procedural or off task activities. Then, statistical discourse analysis (SDA) was used to examine the 
sequential relations predicting lower and higher cognition (Chiu & Koo, 2005). 


19 IFLR 


I. Molenaar 



6 turns ago 5 turns ago 4 turns ago 



Figure 1. Path diagram of standardized final multivariate outcome, multilevel cross-classification of low cognition 
component. Solid lines indicate positive effects. Dashed lines indicate negative effects. Thicker lines indicate larger 
effect sizes. *p < .05, **p < .01, ***p < .001. (Molenaar & Chiu, 2014; reproduced with permission) 

We found that high cognitive, low cognitive, metacognitive and relational activities in recent 
conversation turns were linked to the likelihood of low cognition in a conversation turn (see Figure 1). 
Metacognitive activities in the form of planning (in the previous conversation turn or -1), monitoring (-1), 
evaluating (2 conversation turns ago or -2), monitoring (-2), summarizing (-3) and monitoring (-3) all 
increased the likelihood of low cognition, while orientation (-2) reduced it. Higher cognitive activities in 
either of the last two conversation turns or low cognition in any of the last six conversation turns also 
increased the likelihood of low cognition. Lastly, relational activities in the form of confirming and 
engaging in any of the last two conversation turns increased the likelihood of low cognition. 

This example analyzes temporal characteristics of arrangement of multiple events to understand how 
these events act within the learning process. This type of analysis illustrates how different learning activities 
alternate and fluctuate among collaborating students and emerge into socially regulated learning. The 
findings show recurrent sequential relationships between cognitive, metacognitive and social relational 
activities. Moreover, this analysis indicates that these patterns are rather stable over time. 

Even though these analyses reveal important information about micro-level temporal interaction 
among learning activities, an often received question is: “what do these relations among learning activities 
mean for learning, i.e. which sequences should we encourage with instructional designs?” This question 
embodies the “ holy grail” and has not been addressed yet. Although it is an important question, this inquiry 
clearly indicates the need for a paradigm shift within our field. We need to learn to value results of temporal 
analysis in their own right, providing important information about constructs in learning and instruction and 
taking steps to defining micro level temporal theories of how constructs behave over time. 
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6. Challenges 

Apart from creating the awareness of the need for temporal research questions, there are a number of 
other challenges that need to be addressed to forward temporal analysis in the field of learning and 
instruction. As discussed in section 4, time can be conceptualised differently in our research (Bloome et al., 
2009; Lemke, 2000; Mercer, 2008). A conceptual framework to articulate different dimensions of time to 
frame temporal characteristics and related research questions could enhance conceptual clarity and provide 
ground for in-depth debate about time-related characteristics of individual events in the continues flow of 
events or about the arrangements of multiple events over time. 

Second, although there are many emerging methods such as visualizations (Reimann, 2009), 
sequential lag analysis (Bakeman and Gottman, 1997), statistical discourse analysis (Chiu & Khoo, 2005), 
temporal pattern analysis (Magnusson, 2000), Markov Modeling (Biswas, Kinnebrew & Segedy, 2012), data 
mining (Robero et al. 2010), and dynamic systems (Hollenstein, 2013) used to explore time and order in 
learning processes, we are only starting to explore the commonalities and differences among these methods. 
Understanding about these techniques, as well as which learning and instruction questions can be answered 
by their application, is required. Comparing different methods can enhance our understanding of temporal 
characteristics of constructs in learning and instruction (e.g., via triangulation) and methodological issues 
(e.g., which method is most appropriate for specific research questions?). Collaboration among researchers is 
needed to create guidelines and to work towards a methodological framework for temporal analysis. 

Third, when performing time-related research, we always “cut in time” i.e., we make an artificial 
division in time units. This segmentation of time can be approached differently, that is at the level of 
instructional units, time units or units of time in which a construct is acting homogeneously. For example, 
determining the time window based on the frequency of occurrence of low cognition in the group discourse 
(Molenaar & Chiu, 2014). Choices made about segmentation have important implications for the results, 
and therefore, clear guidelines towards determining time windows should be formulated. 

Fourth, granularity of our time related-research is an issue. The level at which we collect and code is 
often at a micro level capturing very small units, such as events from electronic learning environments or 
utterances in a dialogue. Our theories are usually defined at a macro level, explaining how different 
constructs act. These different levels of granularity between coding and theory are a challenge for meaning 
making. Aggregation of micro level variables to more macro level constructs can be a solution to this issue. 
Yet, as with segmentation, decisions about granularity used in analysis also impacts results profoundly and 
should therefore follow clear procedures to ensure quality standards. Moreover, combinations of different 
research traditions, such as ethnographical approaches and data-mining methods, can help make connections 
between macro level theory and micro level coding. A number of researchers have already indicated the need 
for micro level temporal theories of constructs to support temporal analysis (Azevedo, 2014; Bannert et al. 
2014; Molenaar & Chiu, 2014; Molenaar & Jarvela, 2014; Molenaar et al., 2011; Kuvulja et al. 2014; Winne, 
2014). 

Finally, until now, mainly exploratory studies have been done and there is a request from our 
community to move toward to the Holy Grail, that is to establish that particular temporal characteristics 
contribute to learning performance in particular ways. On the one hand, the Holy Grail will help confirm the 
value of temporal analysis for the field of learning and instruction. Yet, as indicated above, linking these 
analysis to learning performance is challenging. Collaboration among researchers is needed to overcome 
these issues and create guidelines to work towards a uniform approach for event-based methods to enhance 
our understanding of the temporal characteristics of learning and instruction. 
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7. Conclusion 

In the field of learning and instruction, there is an intuitive belief that temporality is important to 
comprehend learning. In order for us, as a field, to make progress in understanding the temporal aspects of 
learning, a number of challenges need to be overcome. The field needs to be aware that temporal analysis 
often departs from the traditional research approach. In order to enhance this advancement, the field must 
embrace a different kind of research question specifically related to temporal aspects of learning and 
instruction. 


Keypoints 

9 Time deserves more attention in learning and instruction research 

9 Temporal analysis entails a paradigm shift addressing a different type of research question 

A conceptual framework of time can support framing temporal characteristics and research 
questions 

9 We need to advance our understanding of methodologies, time segmentation and meaning making 
of temporal analysis 
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